Guessing under source uncertainty

This paper considers the problem of guessing the realization of a finite alphabet source when some side information is provided. The only knowledge the guesser has about the source and the correlated side information is that the joint source is one among a family. A notion of redundancy is first defined and a new divergence quantity that measures this redundancy is identified. This divergence quantity shares the Pythagorean property with the Kullback-Leibler divergence. Good guessing strategies that minimize the supremum redundancy (over the family) are then identified. The min-sup value measures the richness of the uncertainty set. The min-sup redundancies for two examples - the families of discrete memoryless sources and finite-state arbitrarily varying sources - are then determined.Comment: 27 pages, submitted to IEEE Transactions on Information Theory, March 2006, revised September 2006, contains minor modifications and restructuring based on reviewers' comment

Guessing based on length functions

A guessing wiretapper's performance on a Shannon cipher system is analyzed for a source with memory. Close relationships between guessing functions and length functions are first established. Subsequently, asymptotically optimal encryption and attack strategies are identified and their performances analyzed for sources with memory. The performance metrics are exponents of guessing moments and probability of large deviations. The metrics are then characterized for unifilar sources. Universal asymptotically optimal encryption and attack strategies are also identified for unifilar sources. Guessing in the increasing order of Lempel-Ziv coding lengths is proposed for finite-state sources, and shown to be asymptotically optimal. Finally, competitive optimality properties of guessing in the increasing order of description lengths and Lempel-Ziv coding lengths are demonstrated.Comment: 16 pages, Submitted to IEEE Transactions on Information Theory, Special issue on Information Theoretic Security, Simplified proof of Proposition

Decentralized sequential change detection using physical layer fusion

The problem of decentralized sequential detection with conditionally independent observations is studied. The sensors form a star topology with a central node called fusion center as the hub. The sensors make noisy observations of a parameter that changes from an initial state to a final state at a random time where the random change time has a geometric distribution. The sensors amplify and forward the observations over a wireless Gaussian multiple access channel and operate under either a power constraint or an energy constraint. The optimal transmission strategy at each stage is shown to be the one that maximizes a certain Ali-Silvey distance between the distributions for the hypotheses before and after the change. Simulations demonstrate that the proposed analog technique has lower detection delays when compared with existing schemes. Simulations further demonstrate that the energy-constrained formulation enables better use of the total available energy than the power-constrained formulation in the change detection problem.Comment: 10 pages, two-column, 10 figures, revised based on feedback from reviewers, accepted for publication in IEEE Trans. on Wireless Communication

Belief propagation for optimal edge cover in the random complete graph

We apply the objective method of Aldous to the problem of finding the minimum-cost edge cover of the complete graph with random independent and identically distributed edge costs. The limit, as the number of vertices goes to infinity, of the expected minimum cost for this problem is known via a combinatorial approach of Hessler and W\"{a}stlund. We provide a proof of this result using the machinery of the objective method and local weak convergence, which was used to prove the $\zeta(2)$ limit of the random assignment problem. A proof via the objective method is useful because it provides us with more information on the nature of the edge's incident on a typical root in the minimum-cost edge cover. We further show that a belief propagation algorithm converges asymptotically to the optimal solution. This can be applied in a computational linguistics problem of semantic projection. The belief propagation algorithm yields a near optimal solution with lesser complexity than the known best algorithms designed for optimality in worst-case settings.Comment: Published in at http://dx.doi.org/10.1214/13-AAP981 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Guessing Revisited: A Large Deviations Approach

The problem of guessing a random string is revisited. A close relation between guessing and compression is first established. Then it is shown that if the sequence of distributions of the information spectrum satisfies the large deviation property with a certain rate function, then the limiting guessing exponent exists and is a scalar multiple of the Legendre-Fenchel dual of the rate function. Other sufficient conditions related to certain continuity properties of the information spectrum are briefly discussed. This approach highlights the importance of the information spectrum in determining the limiting guessing exponent. All known prior results are then re-derived as example applications of our unifying approach.Comment: 16 pages, to appear in IEEE Transaction on Information Theor

Further Results on Geometric Properties of a Family of Relative Entropies

This paper extends some geometric properties of a one-parameter family of relative entropies. These arise as redundancies when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the Kullback-Leibler divergence. They satisfy the Pythagorean property and behave like squared distances. This property, which was known for finite alphabet spaces, is now extended for general measure spaces. Existence of projections onto convex and certain closed sets is also established. Our results may have applications in the R\'enyi entropy maximization rule of statistical physics.Comment: 7 pages, Prop. 5 modified, in Proceedings of the 2011 IEEE International Symposium on Information Theor